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Preface 


The  purpose  of  this  study  was  to  highlight  the 
deficiency  of  guidance  in  the  area  of  development  and 
acquisition  of  Air  Force  software  user  manuals.  As  the  Air 
Force  continues  to  procure  sophisticated  weapon  systems, 
it's  imperative  that  it  ensures  the  delivery  of  quality 
support  documentation  for  use  by  Air  Force  personnel. 
Unfortunately,  this  problem  was  highlighted  many  years  ago 
with  no  apparent  effect  on  existing  regulations.  I  hope 
this  thesis,  or  spinoff  articles  derived  from  it,  may 
stimulate  the  right  people  to  correct  this  deficiency. 

I  would  like  to  thank  my  thesis  advisor.  Major  John 
Stibravy  for  his  guidance  and  support.  I  would  also  like  to 
thank  Nr.  Art  Munguia  for  contributing  his  time  as  a  reader 
to  this  thesis,  and  acting  as  a  quality  checker  for 
information  presented  on  the  Air  Force  Technical  Order 
system. 


Amy  M.  Baines 
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Abstract 


Because  of  the  high  costs  associated  with  software 
development,  the  Air  Force  has  placed  greater  emphasis  on 
controlling  the  cost  and  quality  of  software  being  procured. 
With  this  increased  emphasis  comes  the  need  to  ensure  that 
the  documentation  procured  to  support  that  software  is  also 
quality  controlled.  This  research  evaluated  the  adequacy  of 
current  regulations  by  comparing  the  process  for  developing 
Air  Force  software  manuals  with  the  process  for  developing 
similar  types  of  user  manuals,  both  within  and  outside  of, 
the  Air  Force. 

The  regulations  guiding  the  development  of  Air  Force 
software  user  manuals  were  first  compared  to  the  regulations 
guiding  the  development  of  Air  Force  Technical  Orders.  The 
requirements  for  software  manuals  were  found  to  be  unclear, 
dispersed  through  numerous  regulations,  and  at  times 
contradictory  from  one  regulation  to  the  next.  The 
requirements  for  Technical  Orders  are  centralized  and 
clearly  organized. 

In  comparing  the  Air  Force  regulations  to  the  best 
commercial  practices  used  by  industry  for  developing 
software  manuals,  it  was  found  that  private  companies  have 
much  more  stringent  requirements  for  ensuring  quality 
documentation . 
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The  research  reconinends  a  revision  and  consolidation  of 
the  regulations  for  software  user  manuals  into  a  separate 
set  of  standards,  similar  to  those  for  Technical  Orders. 
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AN  INVESTIGATION  OF  THE  ADEQUACY  OF  AIR 
FORCE  REGULATIONS  GUIDING  THE  DEVELOPMENT 
AND  PROCUREMENT  OF  SOFTWARE  USER  MANUALS 


I.  Introduction 


General  Issue 

During  the  acquisition  of  new  weapon  systems,  the  Air 
Force  procures  technical  documentation  deemed  necessary  to 
operate  and  support  those  systems.  Most  types  of  technical 
documentation  procured  have  specific  format  and  content 
requirements  specified  in  military  specifications  and 
standards.  The  appropriate  specifications  and  standards  are 
included  in  the  acquisition  contract,  and  thereby  imposed 
upon  the  contractor  for  use  in  developing  and  delivering  the 
documents . 

The  most  important  types  of  technical  documents  are 
user  documents;  these  are  intended  for  use  by  Air  Force 
personnel  on  a  regular  basis  to  operate  or  maintain  the  new 
system.  User  documents  are  applicable  to  both  hardware  and 
software  components  of  a  system.  With  the  increasing  use  of 
sophisticated  computer  programs  in  military  weapon  systems 
comes  a  greater  requirement  to  ensure  the  adequacy  of  the 
documentation  for  operating  and  supporting  those  computer 
systems.  In  the  last  few  years,  the  Air  Force  has  placed 
greater  emphasis  on  the  quality  of  software  being  procured. 


1 


and  has  generated  new  regulations  for  stricter  guidance  and 
control  of  software  development  (16:1).  It  is  important  to 
ensure  that  the  user  documentation  to  support  that  software 
is  also  strictly  controlled  to  provide  quality  documentation 
for  Air  Force  personnel. 

Background 

Generally,  when  a  new  weapon  system  is  procured, 
computer  software  must  be  developed  that  will  be  embedded 
within,  and  is  peculiar  to,  that  new  system.  There  are  many 
varieties  of  documents  which  can  be  procured  to  support 
computer  software.  Some  documentation  is  procured  to 
monitor  the  process  of  software  development.  This  type  of 
documentation  is  used  by  acquisition  officials  to  control 
the  contractor's  schedule  and  development  strategy  for 
software.  Another  type  of  software  documentation,  perhaps 
the  most  important  type,  is  the  documentation  that  is  used 
to  support  software  once  it  has  been  embedded  in  the 
computer  system.  This  includes  manuals  used  by  Air  Force 
personnel  to  operate  and  maintain  the  software  after  the 
system  is  in  the  field. 

In  1986,  the  Air  Force  revised  and  expanded  its 
regulations  on  software  development  and  acquisition,  in  an 
attempt  to  better  regulate  the  process  and  thereby  control 
costs,  which  can  amount  to  a  considerable  portion  of  the 
total  acquisition  cost.  Although  some  areas  of  the  software 
acquisition  process  are  now  well  documented  and 
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comprehensive,  the  requirements  and  processes  for  software 
manual  documentation,  specifically  user  dociunents,  are  still 
dispersed  and  unclear. 

Another  type  of  user  documentation  that  is  common  in 
the  Air  Force  is  Technical  Orders  (TOs)  which  pertain  to  the 
operation  and  maintenance  of  hardware  components  of  a 
system.  The  development  and  procurement  process  for  TOs  is 
well  documented  and  will  be  used  as  a  basis  for  comparison 
with  the  development  process  of  software  user  manuals.  This 
is  an  appropriate  comparison  since  both  types  of  documents 
are  developed  for  use  by  Air  Force  technicians  in  supporting 
new  weapon  systems. 

Problem  Statement 

No  determination  has  been  made  as  to  the  extent  of  Air 
Force  guidance  on  software  user  manual  development,  and  the 
adequacy  of  that  guidance  for  allowing  acquisition  officials 
to  procure  quality  manuals. 

Research  Objective 

The  objective  of  this  research  is  to  determine  the 
adequacy  of  current  Air  Force  regulations  for  the 
development  and  acquisition  of  quality  software  manuals,  and 
then  identify  possible  changes  or  additions  to  those 
regulations  which  could  improve  their  utility.  This 
objective  will  be  accomplished  by  answering  the  following 
research  questions. 
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Research  Questions 


1.  What  are  all  the  regulations  guiding  the 
acquisition  and  developnent  of  Air  Force  software  user 
manuals? 

2.  How  do  the  requirements  for  Air  Force  software  user 
manuals  compare  to  the  requirements  for  Air  Force  Technical 
Orders? 

3.  How  do  the  requirements  for  Air  Force  software  user 
manuals  compare  to  best  commercial  practices  for  developing 
similar  manuals? 

4.  What  improvements,  if  any,  can  be  made  to  the 
current  Air  Force  regulations  guiding  software  user  manual 
acquisition? 

Scope  ,an.d..,Limi.tat.i.ons 

For  the  purposes  of  this  research,  the  phrase  "software 
user  manuals"  will  refer  only  to  software  manuals  intended 
for  use  by  Air  Force  personnel  in  operating  or  maintaining 
software.  This  research  will  examine  all  regulations, 
specifications,  and  standards  applicable  to  the  development 
and  acquisition  of  Air  Force  software  users  manuals,  in 
conjunction  with  new  system  acquisition;  it  will  not  review 
regulations  for  other  DOD  agencies.  This  assumes  the 
computer  software  is  embedded  within  and  peculiar  to  the  new 
system  being  acquired;  therefore,  this  research  also  assumes 
the  development  of  new  manuals  and  will  not  cover  the  use  of 
existing  Commercial-Off -the-Shelf  (COTS)  manuals. 
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Finally,  this  research  is  exploratory  in  nature;  no 
hypothesis  will  be  tested.  The  intent  is  to  evaluate  the 
adequacy  of  Air  Force  regulations  for  software  user  nanual 
development,  and  make  recommendations  based  on  that 
evaluation . 

Benefits  of  the  Research 

The  potential  benefit  of  this  research  is  to  provide 
improved  guidance  for  the  development  and  acquisition  of 
software  user  manuals.  Stricter  control  of  development, 
especially  within  the  review  process  prior  to  acceptance, 
can  increase  the  effectiveness  of  the  manuals  and  thereby 
improve  the  ability  of  Air  Force  personnel  to  operate  and 
maintain  new  software  systems.  These  benefits  could 
directly  influence  unit  effectiveness  and  operating  costs 
for  a  variety  of  systems  already  procured  and  those  to  be 
procured  in  the  future. 

Summary 

Because  of  the  high  costs  associated  with  software 
development,  the  Air  Force  has  placed  greater  emphasis  on 
controlling  the  cost  and  quality  of  software  being  procured. 
With  this  increased  emphasis  on  software  quality  comes  the 
need  to  ensuru  that  the  documentation  procured  to  support 
that  software  is  also  quality  controlled.  This  research 
will  evaluate  the  adequacy  of  current  regulations  by 
comparing  the  process  for  developing  software  manuals  with 
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the  process  for  developing  similar  types  of  user  manuals, 
both  within  and  outside  of,  the  Air  Force. 

Overview  of  the  Thesis 

Chapter  I  has  provided  a  basic  overview  of  the  purpose 
and  scope  of  this  research  effort.  Chapter  II  will  discuss 
the  methodology  that  will  mployed  to  answer  the  research 
questions,  and  analyze  results  of  the  data  gathered. 
Chapter  III  is  a  literature  review  that  identifies  all 
regulations  guiding  development  of  software  user  manuals;  it 
also  compares  this  guidance  to  Air  Force  guidance  on  the 
development  of  technical  orders  and  to  the  best  commercial 
practices  employed  by  civilian  companies  for  developing 
software  type  user  manuals.  The  literature  review  will 
provide  the  basis  for  making  recommendations  for 
improvements  to  the  current  system  of  Air  Force  software 
user  manual  development.  It  also  provides  a  basis  for 
comparison  between  requirements  the  Air  Force  levies  on  its 
contractors  to  produce  military  products,  and  those  levied 
by  commercial  companies  upon  themselves,  for  developing 
similar  products. 

Chapter  IV  will  analyze  the  results  of  the  literature 
review  and  provide  specific  answers  to  the  investigative 
questions  listed  above.  Chapter  V  will  summarize  the 
results  of  the  literature  review  and  analysis,  and  provide 
recommendations  for  changes  to  Air  Force  regulations  guiding 
development  of  software  user  manuals. 
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II.  Mgthodolpqy 


Introdugtign 

This  chapter  will  present  the  exact  method  of  data 
collection  that  will  be  employed  to  answer  the  research 
questions  put  forth  in  Chapter  I.  A  justification  of  the 
methods  chosen  will  be  provided,  as  well  as  the  process  for 
data  analysis. 

Research  Methodology 

As  stated  in  Chapter  1,  this  research  is  exploratory. 

As  such,  all  Investigative  Questions  (#1  through  #4)  will  be 
answered  by  conducting  an  extensive  literature  review  of  Air 
Force  regulations,  military  (or  DOD)  standards,  and 
specifications,  as  well  as  commercial  literature  which  deals 
with  software  manual  or  software  documentation  development. 
Investigative  Question  #1  (What  are  all  the  regulations 
guiding  the  acquisition  and  development  of  Air  Force 
software  user  manuals?)  will  be  answered  by  reviewing  all 
regulations  applicable  to  the  development  and  procurement  of 
Air  Force  software  user  manuals.  Investigative  Question  #2 
(How  do  the  requirements  for  Air  Force  software  user  manuals 
compare  to  the  requirements  for  Air  Force  Technical  Orders?) 
will  be  answered  by  reviewing  all  regulations  applicable  to 
the  development  and  procurement  of  Air  Force  TOs. 
Investigative  Question  #3  (How  do  the  requirements  for  Air 
Force  software  user  manuals  compare  to  best  commercial 
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practices  for  developing  similar  manuals?)  will  be  answered 
by  reviewing  commercial  literature  that  deals  with  %n:iting 
technical  documentation  (in  general),  as  well  as  developing 
and  writing  software  documentation  for  software  users.  In 
most  cases,  this  information  will  be  contained  in  business 
related  periodicals,  technical  journals,  or  organizational 
proceedings,  such  as  International  Technical  Communication 
Conference  Proceedings  (29).  The  documentation  reguired  for 
this  review  was  acquired  from  the  Wright  Research  and 
Development  Center  Technical  Library,  Wright-Patterson  AFB, 
Building  #  22,  the  Air  Force  Institute  of  Technology  (AFIT) 
Academic  Library,  Building  #642,  and  the  Wright  State 
University  Library,  Wright  State  University,  Dayton  Ohio. 

Investigative  Question  #4  (What  improvements,  if  any, 
can  be  made  to  the  current  Air  Force  regulations  guiding 
software  user  manual  acquisition?)  will  be  answered  by 
comparing  the  requirements  for  Air  Force  software  user 
manuals.  Air  Force  TOs,  and  commercial  software 
documentation  in  the  areas  of  document  content/format, 
quality  reviews,  and  distribution/update  control  after 
publication. 

Justification  of  Methodology 

In  his  book.  Business  Research  Methods.  Emory  devotes  a 
chapter  to  the  use  of  secondary  data  sources  as  a  tool  for 
research  (19:135-152).  Emory  states  that  "...secondary  data 
may  be  used  as  the  sole  basis  for  a  research  study" 
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(19:136).  In  discussing  the  various  types  of  secondary 
sources,  he  states  "Government  publications  are  good  sources 
of  information  in  many  topic  areas"  and  "Periodicals  are 
often  the  best  single  source  of  information  for  the  business 
researcher. .  .especially  useful  in  providing  the  most  cxirrent 
information"  ( 19 : 144 ) . 

Validation  of  Method 

In  order  to  use  secondary  data  with  confidence,  the 
researcher  must  ensure  that  the  data  used  is  pertinent, 
accurate,  complete,  and  derives  from  a  reliable  source 
(19:152).  In  order  to  validate  the  data  gathered  for  this 
research,  Hr.  Arthur  Munguia,  AFIT  Associate  Professor, 
School  of  Systems  and  Logistics  and  an  expert  in  the 
acquisition  of  TOs,  was  a  reader  for  this  thesis. 

Summary 

This  chapter  has  discussed  the  method  of  data 
collection  and  analysis  for  this  research.  An  extensive 
literature  review  was  performed  to  compare  the  requirements 
for  developing  Air  Force  software  user  manuals  to  the 
requirements  for  developing  Air  Force  TOs  and  commercial 
software  user  manuals.  The  resulting  data  was  used  to  make 
recommended  improvements/changes  to  current  Air  Force 
regulations  guiding  development  of  software  user  manuals. 

The  next  chapter  is  the  literature  review,  which 
describes  in  detail  all  the  regulations  guiding  the 
development  and  acquisition  of  Air  Force  software  user 
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manuals  and  Air  Force  TOs.  This  chapter  also  explores  the 
best  commercial  practices  currently  being  used  by  industry 
to  develop  user  manuals  for  distribution  with  computer 
products,  or  for  internal  use  by  company  employees. 
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III.  Literature  Review 


Introduction 

This  literature  review  examines  the  extent  of  Air  Force 
policy  guiding  the  development  of  software  user  manuals, 

TOs,  and  commercial  computer  manuals,  through  an  analysis  of 
current  Air  Force  acquisition  requirements,  and  best 
commercial  practices.  This  literature  review  is  intended  to 
answer  the  following  research  questions: 

1.  What  are  all  the  regulations  guiding  the 
acquisition  and  development  of  Air  Force  software  user 
manuals? 

2.  How  do  the  requirements  for  Air  Force  software  user 
manuals  compare  to  the  requirements  for  Air  Force  Technical 
Orders? 

3.  How  do  the  requirements  for  Air  Force  software  user 
manuals  compare  to  best  commercial  practices  for  developing 
similar  manuals? 

Because  the  acquisition  of  Air  Force  software  user 
manuals  is  intricately  tied  to  the  development  of  the 
software  itself.  Section  1  provides  an  overview  of  the  Air 
Force  software  acquisition  process.  Section  2  contrasts  Air 
Force  requirements  for  software  manuals  with  the 
requirements  for  Technical  Orders.  This  section  is 
organized  into  three  subsections:  Content/Format,  Quality 
Reviews,  and  Update/Control.  Within  each  subsection  is  a 
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discussion  of  that  development  area  as  it  pertains  to  both 
Air  Force  soft%fare  user  manuals  and  Air  Force  TOs. 

Section  3  discusses  commercial  practices  for  developing 
computer  user  manuals. 


Discussion  of  the  Literature 

SestiPII- 1 . _ The  Air  Force  Software  Development  Process 

Software  procurement  for  the  Air  Force  is  required  to 
follow  a  specific  development  cycle.  The  cycle  consists  of 
six  sequential  phases,  most  of  which  have  an  associated 
review  or  audit  of  contractor  progress  (16:25).  The  six 
phases  with  associated  reviews  are  summarized  below: 

Software  Requirements  Analysis-  includes 
evaluating  requirements  for  completeness, 
understandability,  validity,  and  consistency  and... is 
followed  by  a  Software  Specification  Review  (SSR)... 

Preliminary  Design  Review-  includes  top-level 
software  design,  critical  lower-level  software 
design,  and... is  followed  by  a  Preliminary  Design 
Review  (PDR)... 

Detailed  Design-  involves  refining  the  top- 
level  software  design  to  define  all  information 
necessary  for  coding. . .This  step  is  followed  by  a 
Critical  Design  Review  (CDR)... 

Coding  and  Unit  Testing-  coding  translates  the 
detailed  design  into  computer  instructions  and  data 
definitions. 

Computer  Software  Component  (CSC)  Integration 
and  Testing-  consists  of  incrementally  integrating 
units  and  components,  and  informally  testing  the 
result. . .This  integration  is  followed  by  a  Test 
Readiness  Review  (TRR)... 

Computer  Software  Configviration  Item  (CSCI) 
Testing-  consists  of  formal  tests  to  verify  that 
each  function  of  the  CSCI. . .satisfies  the  Software 
Requirements  Specif icat ion. . .This  step  is  followed  by 
a  Functional  Configuration  Audit  (FCA)....  (16:25-26) 
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The  six  phases  outlined  are  described  in  Air  Force 
Regulation  (AFR)  800-14,  Lifecycle  Management  of  Computer 
Resources  in  Systems.  AFR  800-14  discusses  the  planning, 
development,  acquisition,  and  support  of  computer  resources 
in  general  terms  (16:1).  More  specific  guidelines  for 
software  development  can  be  found  in  Department  of  Defense 
Standard  (IX)D-STD)  2167A,  Defense  System  Software 
Development .  which  is  referenced  in  AFR  800-14  (16:25). 
These  regulations,  and  others,  will  be  discussed  later  in 
relation  to  the  requirements  for  software  manual 
development . 

Section . 2 1 _ Requirements  for  Software  User  Manuals  vs 

Technical  Orders 

1.  Content/Focmatt 

Software  Manual  Development.  Although  AFR 

800-14  is  supposed  to  provide  general  guidance  for  the 

development  and  support  of  computer  resources,  only  two 

direct  references  to  documentation  exist.  The  first  refers 

to  development  and  can  be  found  under  the  heading  of 

"Support  Documentation": 

Support  documentation  includes  documents 
related  to  system  management,  design,  operation,  and 
maintenance.  All  software  support  documentation  must 
be  delivered  in  sufficient  quality  and  detail  to 
permit  organic  government  support  for  the  life  of  the 
system.  (16:11) 
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The  second  reference  concerns  updating  documentation: 

All  documentation  ( operator /user ,  maintenance, 
programmer,  training,  system/software  specifications) 
will  be  updated  to  reflect  software  changes  and  be  made 
available  concurrent  with  distribution  of  updated  CSCI 
products.  (16:18) 

More  specific  guidance  can  be  found  in  D0D-STD-2167A, 
which  is  intended  to  provide  specific  means  for  establishing 
and  maintaining  quality  in  the  development  of  Air  Force 
software  and  its  related  documentation  (9:iii).  This 
standard  is  important,  because  it  defines  and  limits  the 
types  of  software  manuals  that  can  be  procured,  and  also 
delineates  the  specific  requirements  for  the  development  of 
those  manuals  (9:35). 

When  a  new  contract  is  established,  a  list  is  prepared 
of  all  the  data  items  the  government  expects  the  contractor 
to  deliver.  This  list  is  called  a  Contract  Data 
Requirements  List  (CDRL).  Each  type  of  data  item  delivered 
has  an  associated  Data  Item  Description  (DID)  which 
describes  the  basic  format  and  content  requirements  for  that 
delivery.  DOD-STD  2167A  discusses  data  items  related  to 
software  development  (9:35).  Four  of  the  DIDs  referenced  in 
this  standard  define  software  manuals  for  Air  Force  users. 

The  four  DIDs  are  listed  below  with  short  descriptions 
of  their  intended  use: 
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Conputer  Systea  Operator's  Manual  (CSOM)- 
provides  information  and  detailed  procedures  for 
initiating,  operating,  monitoring,  and  shutting  down 
a  computer  system  and  for  identifying/isolating  a 
malfunctioning  component  in  a  computer  system.  A 
CSOM  is  developed  for  each  computer  system  in  which 
one  or  more  CSCIs  execute.  (8:1) 

Software  User's  Manual  (SUM)-  provides  user 
personnel  with  instructions  sufficient  to  execute 
one  or  more  related  CSCIs.  The  SUM  provides  the 
steps  for  executing  the  software,  the  expected 
output,  and  the  measures  to  be  taken  if  error 
messages  appear.  The  information  required  by  this 
DID  is  directed  to  the  functional  user  of  the  CSCI, 
as  opposed  to  the  operator  of  the  computer  system. 
(34:1) 

Software  Programmers  Manual  (SPM)-  provides 
information  needed  by  a  programmer  to  understand  the 
instruction  set  architecture  of  the  specified  host 
and  target  computers.  The  SPM  provides  information 
that  may  be  used  to  interpret,  check  out, 
troubleshoot,  or  modify  existing  software  on  the 
host  and  target  computers.  (33:1) 

Firmware  Support  Manual  (FSM)-  provides 
information  necessary  to  load  software  or  data  into 
firmware  components  of  a  system.  It  is  equally 
applicable  to  read  only  memory  (ROMs),  Programmable 
ROMs ,( PROMs ) ,  Erasable  PROMs  (EPROMs),  and  other 
firmware  devices.  The  FSM  describes  the  aspects  of 
the  firmware  devices,  support  software,  support 
equipment,  and  the  procedures  required  to  load 
software  into  firmware  devices  to  verify  the  load 
process  and  to  test  the  firmware  device  for  proper 
functioning.  (20:1) 

As  mentioned  earlier,  DIDs  contain  instructions  to  the 
contractor  for  the  basic  format  and  content  requirements  for 
the  data  to  be  delivered.  Each  of  the  manuals  listed  above 
requires  the  same  basic  format:  cover  page,  title  page,  a 
table  of  contents,  scope,  referenced  documents  list,  manual 
specific  procedures,  notes,  and  finally  appendixes  (8:2; 
20:2;  33:2;  34:2).  The  requirements  for  the  manual- 
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specific  procedures  are  broken  down  into  sections  within 
each  DID. 

These  DIDs,  which  average  5  pages  each,  are  the  only 
guidance  documents  that  provide  specific  content 
requirements  for  these  manuals. 

One  document  was  found  that  specifically  addresses 
software  documentation  requirements  (including  those  for 
user  manuals).  Although  titled  An  Air  Force  Guide  to 
Software  Documentation  Requirements,  it's  an  Electronics 
System  Division  (ESD)  dociiment  that  was  produced  by  the 
Mitre  Corporation  of  Bedford  Massachusetts  (35).  The  guide 
discusses  documentation  used  to  monitor  the  software 
acquisition  process  as  well  as  documentation  intended  for 
use  by  Air  Force  personnel,  including  the  Positional 
Handbook  and  Computer  Users  Manual.  For  each  type  of 
document,  the  following  subsections  are  addressed:  Purpose, 
Uses,  Applicability,  Relationship  to  Other  Documents, 
Assessing  Adequacy,  and  Potential  Problem  Areas.  This  guide 
is  by  far  the  most  detailed  document  in  terms  of 
specifications  and  procedures  for  software  documentation. 
Apparently  produced  in  response  to  an  ESD  contract  for 
determining  or  consolidating  Air  Force  requirements  for 
software  documentation  development,  the  guide  provides  the 
following  conclusions: 
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1.  There  is  no  single  source  of  guidance  on 
software  documentation. . .This  guidebook  may  serve 
as  .  consolidation  of  various  Air  Force  sources, 
sximmarizing  standard  data  items.. 

2.  There  is  a  lack  of  guidance  on  the 
reguirements  for,  and  usage  of,  documentation 
related  to  software  and  its  acquisition. 

3.  Another  general  observation  is  the  general 
lack  of  detail  in  the  DIDs.  Coupled  with  the  lack 
of  guidance  on  software  documentation  requirements 
and  the  lack  of  definitions,  this  situation  is 
unfortunate.  (35:133-134) 


Technical  Order  Development.  AFR  8-2,  Air 
Force  Technical  Order  System  contains  official  policy  on  the 
Air  Force  Technical  Order  (TO)  system  (14:1).  The  TO-00-5 
series  of  regulations  further  defines  the  TO  system.  TO-00- 
5-1  gives  an  overview  of  the  whole  Air  Force  TO  system  (15), 
TO-00-5-2  discusses  the  TO  distribution  system  (17),  and  TO- 
00-5-3  provides  acquisition  procedures  for  TOs  (12). 

TO  00-5-1,  Air  Force  Technical  Order  System,  gives  a 
description  of  the  types  of  TOs  in  the  Air  Force  (15:2-1). 
There  are  five  general  categories:  System  and  Equipment  TOs, 
Time  Compliance  TOs,  Methods  and  Procedures  TOs,  Index  Type 
TOs,  and  Abbreviated  TOs  (15:2-7).  The  actual  requirements 
for  the  format  and  content  of  TOs  can  be  found  in  numerous 
military  standards.  Some  of  these  standards  are  general 
nature;  others  are  designed  for  a  specific  type  of  TO,  or 
sections  of  TOs.  A  short  list  of  standards  is  presented 
next  to  illustrate  the  comprehensive  nature  of  the  system 
regulating  TO  development: 
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MIL-M-38784B 


Requirements 

MIL-M-7700C 

EligHt 

MIL-C-38413B 

Air  Refueling  Procedures 

MIL-M-38807A 

Illustrated  Parts  Breakdown 

MIL-M-38797 

Operation  and  Maintenance 

Instructions 

MIL-M-26788C 

Ooeration  and  Maintenance 

Instructions  (Vehicles) 

MIL-C-9927A 

Operational  and  Oraanizational 

Maintenance  Checklists 

MIL-M-38793 

Calibration  Procedures 

MIL-M-38795B 

System  Peculiar  Corrosion  Control 

(13:29-32) 

The  development  of  any  one  TO  will  often  require 
multiple  standards.  In  addition  to  all  the  standards, 
general  policy  dictates  that  all  newly  developed  TOs  must  be 
written  to  a  9th  grade  reading  level  (13:5). 

Software  manuals  are  not  mentioned  under  any  of  the  TO 
category  descriptions  in  TO  00-5-1  (15:2-1-2-6)  and  there 
are  no  standards  dedicated  to  their  development  (12). 

2.  Quality  RgyjgBg 

Software  Manuals.  Military  Standard  (MIL- 
STD)  1521B,  Technical  Reviews  and  Audits  for  Systems. 
Equipments,  and  Computer  Software,  discusses  the  procedures 
to  be  followed  during  the  reviews  and  audits  of  contractor 
progress  which  were  briefly  mentioned  earlier  (11:19-123). 
This  standard  is  organized  by  review  and  contains  specific 
instructions  on  what  material  is  to  be  presented  for  review. 
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and,  in  some  cases,  the  criteria  for  review  (11:19-123). 

The  first  time  software  manuals  are  specifically  referenced 
is  under  the  procedures  section  for  the  Critical  Design 
Review  (CDR)  which  occurs  toward  the  end  of  the  system 
development  cycle  (11:56).  They  are  listed,  along  with 
multiple  other  items,  under  a  heading  which  reads:  "The 
contractor  shall  present  the  following  for  review  by  the 
contracting  agency:"  (11:54).  No  criteria  for  evaluating 
the  manuals  is  provided.  Software  manuals  are  not  mentioned 
again  uintil  the  procedures  for  the  Physical  Configuration 
Audit  (PCA)  are  discussed  (11:82).  The  instructions 
provided  read: 

As  a  minimum,  the  following  actions  shall  be 
performed  by  the  PCA  team  on  each  CSCI  being 
audited: .. .Check  Software  User's  Manual (s).  Software 
Programmer's  Manual,  Computer  System  Operator's 
Manual,  Firmware  Support  Manual...  for  format 
completeness  and  conformance  with  applicable  data 
item  descriptions.  (Formal  verification/acceptance 
of  these  manuals  should  be  withheld  until  system 
testing  to  ensure  that  the  procedural  contents  are 
correct).  (11:82) 

MIL-STD  1521B  isn't  the  only  document  which  addresses 
software  review,  or  quality  issues.  Military  Specification 
(MIL-S)  52779A,  Software  Quality  Assurance  Program 
Requirements .  mandates  the  establishment  of  a  quality 
assurance  program,  and  associated  plan,  to  ensure  new 
software  complies  with  contract  requirements  (10:1).  None 
of  the  four  software  manuals  discussed  earlier  are 
specifically  mentioned.  There  is,  however,  a  requirement  to 
include  in  the  quality  assurance  plan  the  procedures  which 
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will  be  used  to  assure  software  documentation  complies  with 
contractually  required  standards  (10:3).  No  recommendations 
are  made  as  to  what  types  of  procedures  are  appropriate.  It 
is  apparently  left  to  the  discretion  of  the  specific  Air 
Force  acquisition  agency  to  determine  the  type  and  extent  of 
procedures  to  be  imposed  on  the  contractor. 

Technical  Orders  (10).  TO  00-5-3,  Air  Force 
Technical  Manual  Acquisition  Procedures,  outlines  all  the 
steps  required  for  TO  development  through  each  stage  of 
program  acquisition.  The  process  involves  a  conference 
dedicated  to  TO  development,  multiple  reviews  to  monitor 
contractor  progress,  a  validation  of  the  manuals  by  the 
contractor,  and  a  verification  process  performed  by  the  Air 
Force.  Specific  procedures  and  review  criteria  are 
presented  for  each  step  of  the  process.  In  addition,  there 
are  a  number  of  required  planning  documents  (both  Air  Force 
and  contractor  developed),  which  outline  the  TO  acquisition 
strategy . 

Software  manuals  are  not  mentioned  in  the  descriptions 
of  this  review  cycle. 

3.  Update /Control.  (18) 

Software  Manuals.  Air  Force  Technical  Order 
00-5-16,  Computer  Program  Identification  Numbering  fCPIN^ 
System.  Software  Managers  Manual,  describes  the  CPIN  system 
and  provides  guidance  on  its  use.  The  CPIN  system  is  a  new 
effort  in  the  Air  Force  to  treat  computer  programs  as 
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configuration  items.  Each  CPCI  will  be  assigned  a  CPIN 
number  for  distribution  and  tracking  within  the  Air  Force 
system.  Instructions  for  the  control  of  software  manuals 
are  described  as  follows: 

USER  DOCUMENTATION.  User  documentation  includes 
users  manuals,  test  equipment  manuals,  and  user 
maintenance  manuals  which  are  required  by  the 
user  for  operational  use,  checkout,  installation, 
troubleshooting,  and  loading  of  the  CPCI.  User 
dociimentation  will  not  be  assigned  a  CPIN  but  will  be 
assigned  an  appropriate  technical  order  number  and 
will  be  managed  by  the  Technical  Order  System  as 
applicable.  User  documentation  will  reference  the 
applicable  CPCI  by  the  appropriate  CPIN.  (18:3>2) 


Technical  Orders.  TOs  are  distributed  and 
controlled  according  to  the  policies  and  procedures  of  TO 
00-5-2,  Technical  Order  Distribution  System  (17).  The  Air 
Force  TO  distribution  system  is  managed  by  Oklahoma  City  Air 
Logistics  Center  (OC-ALC)  (14:2-1),  the  same  base  that 
manages  the  CPIN  system  discussed  earlier  (18:1-1). 


Section  3. _ Commercial  Practices  for  Developing 

Software  Manuals 

"When  the  discussion  turns  to  commercial  software,  one 
usually  hears  compliments  for  what  the  software  does  but 
criticisms  for  its  documentation"  (27:355).  One  of  the 
reasons  for  the  above  is  that  currently  there  is  no 
accepted,  documented  standard  for  producing  software  user 
manuals.  There  is  an  organization  designed  to  establish 
standards  for  all  varieties  of  commercial  practices,  the 
American  National  Standards  Institute  (ANSI).  A  1988 
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article  from  the  Journal  of  _Technicial  _Writina  _and 
Communication  states: 


Such  a  documentation  framework  will  result 
primarily  from  the  efforts  of  the  technical 
committees  of  the  various  standards  organizations. 

For  example,  the  ANSI  Accredited  Standards 
Technical  Committee  (X3K1)  on  Project  Documentation 
is  attempting  to  develop  a  standard  for  user 
documentation  of  any  type  of  software  product 
designed  to  be  sold  commercially.  (27:357) 

As  of  this  writing,  however,  no  such  standard  exists 

for  developing  software  user  manuals  (7).  This  lack  of 

specific  guidance  is  often  quoted  as  a  serious  problem  in 

manual  development  (27:356),  and  many  companies  have  begiui 

producing  their  own  self  imposed  standards  (25;  28;  32). 

The  absence  of  specific,  industry  wide  procedures  makes 

it  difficult  to  define  "best  commercial  practices."  The 

infoirmation  that  follows  then,  is  the  recommendations  and 

views  of  various  companies  or  private  sources  regarding 

their  ideas  of  what  constitutes  best  practices  for 

developing  software  user  manuals. 

It  is  generally  agreed  that  quality  user  manuals  are 

essential  for  promoting  the  sale  of  a  software  product.  "If 

a  company's  publications  do  not  work  well,  then  its 

customers  may  never  find  out  how  really  well  the  product 

works"  (1:68).  Although  this  fact  would  seem  obvious 

enough,  attention  to  the  importance  of  quality  software 

documentation  is  a  fairly  recent  trend  (27:355). 

Historically,  software  documentation  was  often  poorly 

organized,  incomplete,  and  badly  written  (27:355).  One 
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software  naintainer  was  quoted  in  a  Coroputerworld  article 
as  saying:  "I'm  going  to  assume  this  is  typical  maintenance 
documentation  -  not  worth  the  paper  it's  printed  on" 

(21:90). 

Besides  the  lack  of  accepted  standards,  there  are  a 
number  of  other  factors  which  can  contribute  to  poorly 
developed  docvimentation .  One  area  of  potential  problems  is 
assigning  the  dociimentation  process  to  the  tnrong  people. 

The  development  of  the  software  feeds  into  the  documentation 
development;  it's  important  that  technical  writers 
understand  the  logic  and  functioning  of  the  software,  but 
they  also  need  good  writing  skills  to  transfer  that 
understanding  to  new  users  (1:66).  A  skilled  software 
engineer  doesn't  necessarily  make  a  good  technical  writer, 
and  a  good  writer  can't  write  effectively  about  processes  he 
doesn't  understand. 

Just  as  an  engineer  must  understand  material 
properties. . .a  competent  technical  writer  should 
understand  how  to  use  rhetorical  devices  to  achieve  the 
most  elegant  and  proper  solutions.  Understanding 
grammar,  its  uses  and  structures,  is  basic  to  all  good 
writing.  Since  most  technical  writers  are  learning 
with  each  new  assignment,  there's  no  time  to  be 
learning  the  basics,  too.  (1:66) 

Ensuring  good  documentation  then,  begins  with  providing 
people  who  have  the  right  skills  for  the  job.  Recognition 
of  this  fact  has  increased  emphasis  on  technical  writing  as 
a  respected  profession,  and  is  increasing  the  availability 
of  skilled  writers. 
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Writing  software  documentation,  which  often  has 
been  relegated  to  programmers  as  an  afterthought, 
is  getting  some  recognition  as  a  field  in  its  o«m 
right.  In  recent  years,  many  universities  and 
colleges  have  introduced  technical  vnriting 
programs.  (23:106) 

Another  crucial  ingredient  for  producing  quality 
manuals  is  management  support.  "Executives  must  be 
convinced  that  professional  publications  are  significant  to 
the  quality  of  the  company's  image"  (1:63).  Without 
emphasis  and  support  from  upper  management,  the  necessary 
resources  (people,  time,  money)  won't  be  devoted  to  the 
project,  and  quality  will  suffer. 

Having  determined  the  right  people  for  developing  the 
software  documentation,  and  securing  management  support,  the 
first  step,  prior  to  actual  development,  is  to  perform  a 
requirements  analysis  (4:362).  A  requirements  analysis  is 
the  starting  point  for  determining  the  type  of  information 
that  will  be  included  in  the  manual,  as  well  as  the  design 
of  the  final  document;  essentially  a  plan  of  attack  for  the 
whole  development  effort,  it  is  extremely  important  for 
producing  user-centered  documentation.  In  its  most  basic 
form,  a  requirements  analysis  includes  a  determination  of 
the  types  of  documents  the  user  will  require  (Reference 
Manuals,  Users  Guides,  Operations  Manuals)  (32),  task 
analysis:  the  tasks  required  of  the  users  (22),  the  media  of 
presentation  (printed  vs  online)  (4:362),  as  well  as 
audience  analysis:  the  skill  level  of  the  intended  users. 
Within  the  requirements  analysis,  the  two  most  involved  and 
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critical  areas  to  document  success  are  task  analysis  and 
audience  analysis. 

Task  analysis  is  the  phase  where  the  system  orientation 
of  the  software  is  translated  into  the  user  orientation 
required  for  the  documentation.  Task  analysis  should  answer 
the  following  types  of  questions: 

Who  performs  the  task? 

What  actions  begin  each  task? 

What  are  the  specific  steps  involved  in  performing 
the  task? 

What  actions  end  each  task? 

Are  there  any  variations  in  hardware  or  in  the 
general  environment  in  which  the  task  takes  place  that 
would  alter  it?  (6:MG-55) 

Audience  analysis  is  a  special  area  of  concern  because 
of  its  applicability  to  all  types  of  user  manual 
development.  Audience  analysis  includes:  determining  the 
end  user's  skills  (expertise  with  computer  operations, 
educational  level  and  corresponding  reading  level), 
motivation  for  using  the  software  (objective,  goals),  and 
frequency  of  use  (intend  to  increase  skills  vs  infrequent 
reference)  (6:MG-56).  "Audience  analysis  will  have  a  major 
impact  on  both  system  and  document  design"  (4:364). 

During  the  actual  development  of  software  manuals, 
there  are  two  main  concerns:  design  (or  format)  of  the 
manuals,  and  the  usability  of  the  document.  Quality 
software  documentation  will  be  designed  with  the  user  in 
mind,  and  based  on  the  results  of  the  requirements  analysis. 
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Although  specific  design  considerations  will  be  a  result  of 
the  type  of  manual  produced,  some  general  distinctions  can 
be  made  between  traditional,  product-centered  documents  and 
the  more  recent,  user-centered  documents  in  terms  of  design 
(2).  Tables  1  and  2  contrast  the  two  approaches. 


Table  1.  Characteristics  of  Product  Centered 
Documentation  (2:WE-45) 


Characteristic 


Expression 


Organization  of  Books 
amd  Chapters 


Number  of  Pages  in 
the  Documentation 


The  users  have  similar  tasks 
to  perform  and  technical 
backgrounds:  therefore,  product 
determines  organization. 

High  page  counts  are  common. 
White  space  is  rare. 


Headings  and  Labels 


Headings  are  based  on  what  the 
product  does. 


Size  of  Units  of  A  single  unit  may  be  three  or 

Information  pages  in  length. 


Use  of  Devices  for  Charts,  when  provided,  include 

Quick  Access  information  on  several  issues. 


Design  of  Steps  in 
A  Procedure 


Procedures  consist  of  any 
number  of  steps.  User  actions 
and  the  response  of  software 
are  numbered. 


User-centered  documentation  is  task  oriented,  and  has  been 
gaining  popularity  for  writing  software  manuals.  Studies 
about  the  benefits  of  task  oriented  procedures  have 
concluded  that  "task  oriented,  step-by-step  information  is 
essential  when  users  are  unfamiliar  with  the  projects" 
(26:362).  Task  oriented  information  appears  to  increase 
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overall  productivity  for  the  user,  while  decreasing  error 
rate  and  the  level  of  dissatisfaction  with  the  product  (26). 


Table  2.  Characteristics  of  User  Centered 
Documentation  (2:WE-45) 

Characteristic  Expression 


Organization  of  Books 
and  Chapters 


Number  of  Pages  in 
the  Documentation 


Headings  and  Labels 


The  users  have  different  tasks 
to  perform  and  technical  back¬ 
grounds:  therefore,  the  users' 
tasks  determine  organization. 

Writers  attempt  to  use  charts 
to  minimize  page  count  and  to 
provide  white  space. 

Labels  are  based  on  what  the 
user  does. 


Size  of  Units  of 
Information 


Use  of  Devices  for 
Quick  Access 


Design  of  Steps  in 
A  Procedure 


Information  is  broken  into 
small  units,  each  under  one 
page  in  length. 

Charts  are  a  commonly  used 
device.  Charts  contain  only 
information  relevant  to  the 
user's  immediate  action. 

Procedures  are  subdivided  so 
that  each  subprocedure  consists 
of  under  seven  steps.  Only 
user  actions  are  numbered. 


Usability  seems  to  be  an  all  encompassing  term; 
generally  referring  to  how  easily  the  user  is  able  to  access 
and  understand  the  Information  presented  in  the  document 
(36:120).  Factors  affecting  usability  are  numerous;  some 
examples  include:  sentence  structure,  grammar  and 
punctuation,  tone,  voice,  tense,  word  choice  (difficulty  vs 
accuracy),  logical  organization,  relevance,  sequence,  and 
balance  (29).  The  correct  application  for  some  of  these 
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factors,  such  as  word  choice,  sequence,  voice,  and  tone,  is 
determined  by  the  characteristics  of  the  user  and  should  be 
obtainable  from  the  results  of  the  audience  analysis.  The 
other  factors  are  determined  by  the  author's  writing  and 
grammar  skills;  providing  skilled  technical  writers  will 
help  decrease  usability  problems. 

Another  area  of  vital  concern,  and  potential  problems, 
is  performing  quality  reviews  of  the  manual.  One  of  the 
greatest  potential  problems  involved  with  quality  reviews  is 
the  time  required  to  perform  them.  Depending  on  the  extent 
of  the  review  cycle  planned,  as  well  as  the  size  of  the 
documents  being  produced,  the  cycle  can  consume  a  good  deal 
of  the  development  schedule.  The  reviews  require  not  only 
the  writer's  time,  but  also  contributions  from  the 
developers,  quality  assurance  personnel,  and  possibly  the 
users.  The  amount  of  cooperation  required  with  so  many 
participants  can  also  create  a  great  deal  of  conflict  and 
personnel  problems. 

At  all  companies,  the  politics  of  documentation 
can  become  especially  heated  during  review  cycles,  when 
publication  drafts  must  be  critiqued  by  developers, 
marketing  staff,  customer  service  personnel,  quality 
controllers,  and  lawyers,  any  or  all  of  whom  won't 
agree  with  parts  of  the  publication  or  with  each  other 
and  few  of  whom  will  acknowledge  limits  to  their 
critical  ability.  (1:68) 

This  potential  conflict  and  the  required  assurance  of 
sufficient  time  for  performing  quality  reviews,  highlights 
the  importance  of  management  support  and  a  team  approach  to 
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the  review  cycle  in  order  to  assure  the  ultimate  success  of 
the  documentation  process. 

The  review  of  the  documentation  usually  gets  put 
at  the  bottom  of  the  progranmter ' s  priority  list.  Many 
weeks  past  the  requested  date,  the  iinriter  will  finally 
receive  the  corrected  drafts. 

The  most  egregious  cause  for  delay  occurs  when 
management  has  budgeted  no  time  for  the  review  of 
documentation . 

The  second  most  flagrant  cause  of  delay  occurs 
when  management  has  budgeted  the  time  for  reviewing 
documentation,  but  uses  the  time  for  programming 
because  the  project  is  behind  schedule.  (30:26) 

Besides  management  support,  another  important 

consideration  for  ensuring  the  success  of  the  documentation 

review  process  is  the  use  of  a  publication  plan  for 

organizing  and  controlling  the  whole  document  development 

process.  This  publication  plan  can  help  eliminate 

scheduling  problems  and  increase  cooperation  among  the 

involved  departments. 

The  publications  plan  is  used  to  describe  the 
publications  that  will  be  produced  for  the  specified 
product.  The  description  should  include  an  annotated 
list  of  the  publications  to  be  written,  what 
dependencies  govern  their  production,  an  outline  (as 
detailed  as  possible)  of  each  publication,  and  a 
schedule  for  their  development.  The  publications  plan 
also  should  include  schedules  for  each  version  of  each 
manual ,  including  dates  of  distribution  for 
review,  review  meetings,  the  final  date  to  submit 
comments  and  changes  to  the  «n:iter,  and  the  date  each 
version  will  be  finished,  to  the  printer,  and  ready  to 
ship.  (31:WE-148) 

The  depth  of  quality  reviews  can  vary  greatly  depending 
on  the  philosophy  of  the  company,  and  the  intent  of  the 
documentation;  they  range  from  in-house  reviews  of  the  final 
product  to  user  input  throughout  the  development  process, 
with  usability  testing  before  publication. 
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In-house  inspection  of  draft  manuals  is  the 
traditional,  most  widely  practiced  form  of  quality  reviews, 
and  is  critical  to  ensure  the  accuracy  of  the  documentation. 
Durin'?  an  inspection,  the  technical  accuracy  of  the 
documentation  is  checked  by  comparison  with  the  software 
code,  and  any  defects  noted  for  correction.  If  performed 
early  in  the  development  process,  this  inspection  can 
drastically  reduce  costs.  Given  a  cost  of  $X  dollars  to 
correct  errors  during  development,  it  will  cost  ten  times 
that  amount  ($10X)  to  correct  the  deficiencies  at  the 
quality  review  stage,  and  one  hundred  times  that  amount 
($100X)  if  corrected  after  the  document  has  been  published 
(3;WE-52) . 

Recent  emphasis  on  more  user  centered  documentation  is 
increasing  the  popularity  of  including  usability  testing  in 
the  development  of  documentation. 

In  its  strictest  sense,  usability  testing  is  the 
analysis  of  a  product  to  determine  how  useful  it  is  to 
its  targeted  customer. . .the  testing  of  a  document  by  a 
user  to  determine  if  that  document  meets  the  needs  of 
that  user.  (6:WE-154) 

There  are  several  ways  to  conduct  usability 
testing,  among  which  are: 

*  Silent  User 

Video  tape  the  customers  as  they  use  the 
documentation . 

*  Structured  Interview 

Sit  with  the  customers  as  they  use  the  documentation 
and  ask  them  questions. 

*  Verbal  Protocol 

Ask  the  customer  to  think  aloud  as  they  use  the 
documentation  while  video  and  audio  taping  them. 


30 


*  Verbal  Protocol  After  Talk 

Ask  the  customer  questions  about  the  documentation 
after  they  have  used  it. 

*  Validation  Laboratory 

Have  an  agreement  with  a  university  to  provide  space 
and  participants  to  test  the  documentation;  video 
and  audio  tape  the  tests.  (5:MG-58) 

Some  of  the  issues  associated  with  usability  testing  include 

schedule/time  constraints,  added  costs,  locating  suitable 

reviewers  to  use  the  documentation,  acceptable  ranges  of 

usability  scores,  measurement  mechanisms,  and  audit  trails 

(5:WE-151).  When  determining  the  extent  of  quality  review 

that  should  be  performed  for  a  set  of  documentation,  these 

issues,  along  with  their  associated  costs,  should  be  weighed 

against  the  potential  benefits  of  usability  testing. 

Usability  testing. . .does  the  following: 

*  Validates  or  invalidates  the  data  gathered  during 
audience  analysis 

*  Validates  or  invalidates  the  content  as  well  as  the 
design  paradigm  of  the  review  draft 

*  Allows  the  customer  to  input  directly  to  the 
document  and  perhaps  to  improve  the  design 

*  Provides  a  mechanism  for  assessing  the  audience 
feedback  and  for  maintaining  the  currency  of  the 
documentation 

*  Gives  invaluable  insight  into  user  psychology 

*  Allows  measurement  not  only  of  how  customers  use  the 
documentation,  but  at  what  level  of  effort 

*  Supplies  a  tool  to  measure  precisely  where  the 
document  succeeds  and  where  it  fails  (5:MG-57) 

Without  regard  for  cost,  the  most  stringent  and 
effective  (in  terms  of  accuracy)  document  review  cycle  would 
be  the  one  pictured  in  Figure  1. 
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Figure  1.  Documentation  Review  Cycle 


The  last  issue  of  software  documentation  that  will  be 
discussed  is  the  updating,  or  maintenance  process  for 
documentation.  As  compared  to  the  other  issues  discussed  so 
far,  this  topic  is  the  least  discussed  in  the  literature; 
very  little  information  is  available  on  the  details  of  how 
the  commercial  industry  treats  this  aspect  of  software 
documentation . 

Software  documentation  maintenance  is  a  myth  that 
everyone  believes  in  but  almost  no  one  ever  does... only 
a  handful  of  corporations  ever  do  a  decent  job  of 
chronicling  changes,  updates,  and  improvements  to  their 
code.  (21:87) 
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One  particular  article  was  found  that  very  thoroughly 
addressed  the  issue  of  documentation  maintenance.  The  key 
element  identified  was  the  idea  of  planning  the  maintenance 
process  before  publication.  This  planning  involves  creating 
a  schedule  of  maintenance  activities  and  the  production  of  a 
Maintenance  Guide  by  the  developer  (23).  "The  Maintenance 
Guide  is  a  valuable  tool  that  provides  all  of  the 
information  you  need  to  maintain  the  document"  (23:WE-135). 
The  important  elements  of  the  guide  include:  a  description 
of  the  background  information  for  the  document,  the  strategy 
and  requirements  for  maintenance,  the  location  of  materials 
used  to  produce  the  documentation,  the  hardware  and  software 
used  to  produce  the  documentation,  style  guidelines  of  the 
document,  the  production  process  used,  a  distribution  list 
of  all  recipients,  and  a  record  of  maintenance  actions  (23). 
This  Maintenance  Guide  could  be  a  valuable  tool  for 
organizing  and  controling  the  maintenance  process  for 
software  documentation. 

Summary 

This  chapter  has  summarized  an  extensive  literature 
review  which  compares  the  requirements  for  developing  Air 
Force  software  user  manuals  to  the  requirements  for 
developing  Air  Force  TOs  and  commercial  software  user 
manuals.  Section  1  of  this  chapter  briefly  discussed  the 
Air  Force  software  acquisition  process,  and  was  intended  to 
serve  as  an  overview  to  provide  better  understanding  of  the 
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software  manual  acquisition  process,  presented  later  in  this 
chapter.  Section  2  of  this  chapter  contrasted  the 
requirements  for  Air  Force  software  manuals  to  the 
requirements  for  Air  Force  TOs  in  three  general  areas: 
Content/Format,  Quality  Reviews,  and  Update/Control.  The 
final  portion  of  the  chapter.  Section  3,  presented  a  wide 
variety  of  ideas  and  recommendations  on  the  various  factors 
influencing  software  manual  development  within  private 
industry . 

The  next  chapter.  Chapter  IV,  will  analyze  the  results 
of  the  literature  review  presented  in  Chapter  III  to  answer 
the  following  research  questions: 

1.  What  are  all  the  regulations  guiding  the 
acquisition  and  development  of  Air  Force  software  user 
manuals? 

2.  How  do  the  requirements  for  Air  Force  software  user 
manuals  compare  to  the  requirements  for  Air  Force  Technical 
Orders? 

3.  How  do  the  requirements  for  Air  Force  software  user 
manuals  compare  to  best  commercial  practices  for  developing 
similar  manuals? 
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IV.  Findings 


Introduction 

This  chapter  will  analyze  the  results  of  the 
information  presented  in  the  Literature  Review  of  Chapter 
III,  to  answer  the  following  research  questions: 

1.  fRiat  are  all  the  regulations  guiding  the 
acquisition  and  development  of  Air  Force  software  user 
manuals? 

2.  How  do  the  requirements  for  Air  Force  software  user 
manuals  compare  to  the  requirements  for  Air  Force  Technical 
Orders? 

3.  How  do  the  requirements  for  Air  Force  software  user 
manuals  compare  to  best  commercial  practices  for  developing 
similar  manuals? 

Findings  For  Resaargb 

Research  Question  #1:  What  are  all  the  regulations 
guiding  the  acquisition  and  development  of  Air  Force 
software  user  manuals? 

There  are  very  few  regulations  guiding  the  development 
of  Air  Force  software  manuals.  AFR  800-14,  Lifecycle 
Management  of  Computer  Resources  in  Systems,  indirectly 
affects  software  manual  development  by  providing  general 
guidance  on  the  development  of  the  software  to  which  it 
pertains.  The  regulation  only  provides  two  direct 
requirements  for  software  documentation:  that  it  be 


35 


sufficiently  detailed  to  permit  organic  support,  and  that 
documentation  is  updated  to  reflect  software  changes  as  the 
change  is  released  (16). 

DOD-STD-2167A,  Defense  System  Software  Development, 
addresses  software  manuals  more  directly  by  delineating  the 
types  of  manuals  that  can  be  procured  and  listing  the 
respective  number  of  the  Data  Item  Description  (DID)  which 
describes  its  purpose  and  format  (9).  There  are  only  four 
user  type  manuals  listed  in  DOD-STD-2167A:  Computer  System 
Operator's  Manual,  Software  User's  Manual,  Software 
Programmer's  Manual,  and  the  Firmware  Support  Manual  (9). 
There  is  a  five  page  DID  associated  with  each  manual.  Each 
DID  has  the  same  basic  format:  cover  page,  title  page,  table 
of  contents,  scope,  manual-specific  procedures,  and 
appendices;  only  the  "manual-specific  procedures"  section 
differs  from  DID  to  DID  (8;  20;  33;  34). 

In  summary,  the  two  regulations  discussed  above,  AFR 
800-14,  and  DOD-STD-2167A,  along  with  the  five  DIDs,  are  the 
only  documents  guiding  the  content  of  software  user 
manuals. 

Requirements  for  quality  reviews  are  even  less 
stringent  than  those  for  content.  MIL-STD-1521B,  Technical 
Reviews  and  Audits  for  Systems.  Equipments,  and  Computer 
Software .  references  software  user  manuals  twice.  The  first 
reference  requires  that  the  manuals  be  reviewed  at  the 
Critical  Design  Review  (CDR)  for  software  development,  but 
no  evaluation  criteria  are  provided  for  use  in  evaluating 
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the  manual  (11).  Are  the  manuals  to  be  checked  against  the 
software,  or  against  the  requirements  of  the  DID?  Who  in 
the  Air  Force  should  be  reviewing  the  manuals:  users, 
program  office  personnel,  or  engineers?  No  specific 
guidance  on  these  issues  is  provided.  The  second  reference 
is  a  little  more  substantial,  requiring  review  of  the 
manuals  during  the  Physical  Configuration  Audit  (PCA)  to 
ensure  "format  completeness  and  conformance  with  appliceUale 
data  item  descriptions"  (11:82). 

In  addition  to  MIL-STD-1521B,  MIL-S-52779A,  Software 
Quality  Assurance  Program  Requirements,  is  concerned  with 
quality  assurance  of  software  and  its  related  issues  (10). 
MIL-S-52779A  mandates  a  quality  assurance  program,  and 
« sso^dated  plan  to  ensure  compliance  of  the  delivered 
software  products  with  the  specified  requirements  of  the 
contract.  Although  the  four  user  manuals  mentioned  in  DOD- 
STD-2167A  aren't  specifically  referenced,  there  is  a 
requirement  to  include  software  documentation  review 
procedures  in  the  quality  assurance  plan  prepared  by  the 
acquisition  agency  (10:3).  Once  again,  no  procedures  are 
provided,  nor  any  recommendations  as  to  where  to  locate 
appropriate  procedures  or  review  criteria.  It's  apparently 
up  to  the  Air  Force  acquisition  personnel  to  use  the 
contract  requirements  to  set  their  own  procedures  and 
corresponding  stringency  measures  when  reviewing  these 
manuals. 
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The  area  of  update  and  control  for  software  manuals  is 
no  better.  Although  the  software  systems  they  support  are 
well  controlled,  the  requirements  for  the  user  manuals  are 
very  confusing.  Software  programs  are  now  managed  under  Air 
Force  Technical  Order  TO-00-5-16,  Computer  Program 
Identification  Numbering  fCPIN)  System.  Software  Managers 
Manual .  which  assigns  identification  numbers  to  the  software 
for  distribution  and  tracking  (18).  The  requirements  for 
the  user  manuals  are  stated  as  follows: 

USER  DOCUMENTATION.  User  documentation. . .will  not 
be  assigned  a  CPIN  but  will  be  assigned  an  appropriate 
technical  order  number  and  will  be  managed  by  the 
Technical  Order  System  as  applicable.  User 
documentation  will  reference  the  applicable  CPCI  by  the 
appropriate  CPIN .  ( 18 : 3-2 ) 

The  above  statement  seems  to  have  been  thrown  in  to 

compensate  for  the  lack  of  direction  for  controlling 

software  user  manuals.  It  is  apparent  it  wasn't  an 

organized  effort  to  improve  the  deficiencies,  since  the  term 

"as  applicable"  has  no  reference  as  to  what  portions  of  the 

Technical  Order  System  are  in  fact,  applicable.  There  are 

no  specific  references  within  the  TO  regulations  to  user 

manuals;  nor  are  there  any  references  within  the  software 

regulations  to  the  use  of  TO  regulations  for  software 

documentation . 

In  summary,  the  regulations  guiding  quality 
requirements,  or  update  and  control  of  software  user  manuals 
are  dispersed,  unclear,  and  disorganized.  This  fact  was 
recognized  many  years  ago  in  a  document  produced  for 


38 


Electronics  System  Division  through  a  contract  with  Mitre 
Corporation  of  Massachusetts.  The  document,  titled.  An  Air 
Force  Guide  to  Software  Documentation  Requirements, 
attempted  to  consolidate  and  clarify  the  exact  requirements 
for  procuring  software  user  documentation  (35).  Dated  1976, 
the  document  provided  these  conclusions  (amongst  others): 

1.  There  is  no  single  source  of  guidance  on 
software  documentation. . .This  guidebook  may  serve 
as  a  consolidation  of  various  Air  Force  sources, 
summarizing  standard  data  items.. 

2.  There  is  a  lack  of  guidance  on  the 
requirements  for,  and  usage  of,  documentation 
related  to  software  and  its  acquisition. 

3.  Another  general  observation  is  the  general 
lack  of  detail  in  the  DIDs.  Coupled  with  the  lack 
of  guidance  on  software  documentation  requirements 
and  the  lack  of  definitions,  this  situation  is 
unfortunate.  (35:133-134) 

Although  its  stated  purpose  was  to  act  as  a  "guidebook" 
for  consolidating  software  documentation  requirements,  no 
references  to  it  were  ever  found  in  any  of  the  regulations. 
Although  no  longer  applicable  because  many  of  the 
regulations  and  all  of  the  DIDs  have  been  revised  since  its 
%^iting,  the  problems  it  reported  still  exist  today. 
Obviously,  the  conclusions  it  provided  went  unheeded  since 
all  of  the  regulations  discussed  in  this  paper,  which  guide 
development  of  software  manuals,  were  written  after  1979, 
three  years  after  the  paper  was  produced. 
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Research  Question  #2t  How  do  the  requirements  for  Air 
Force  software  user  manuals  compare  to  the  reauirements_for 
Air  Force  Technical  Orders? 

In  each  of  the  areas  reviewed:  content,  quality  review, 
and  control,  the  regulations  guiding  TO  development  are  much 
more  extensive  and  better  organized  than  the  regulations  for 
software  user  manuals.  The  requirements  for  TO  development 
are  primarily  consolidated  in  the  TO-00-5  series  of 
regulations,  with  individual  military  standards  dedicated  to 
almost  every  type  of  TO  developed  for  the  Air  Force.  There 
is  a  set  quality  review  policy  for  all  TOs,  which  requires 
reviews  throughout  development,  as  well  as  a  validation 
(contractor  checks  TO  against  equipment)  and  verification 
(user  testing  for  accuracy  and  readability).  Specific 
review  criteria  and  controls  are  established  for  each  step 
of  this  review  process.  There  is  also  a  standard  document 
for  inclusion  on  all  contracts  requiring  TO  development; 
Technical  Manual  Contract  Requirements  86-01  aids  the 
acquisition  agency  in  ensuring  all  needed  standards  are 
included  in  the  contract  (13).  In  addition,  all  TOs  are 
required  to  be  written  to  a  9th  grade  reading  level. 

Distribution  and  control  of  TOs  is  also  well  organized 
and  well  documented;  TO-00-5-2  lays  out  all  requirements  and 
procedures  for  numbering,  distributing,  and  updating  TOs. 

The  whole  TO  system  is  an  antithesis  of  the  scattered 
regulations  guiding  software  user  manual  development. 
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The  coninercial  industry  faces  the  sane  problen  as  the 
Air  Force  when  it  cones  to  developing  software  user  nanuals: 
there  is  no  single  standard  or  reference  guiding  their 
developnent.  Unlike  acquisition  agencies  within  the  Air 
Force,  however,  private  conpanies  have  the  ability  to 
institute  new  procedures  as  they  see  fit,  to  inprove  the 
quality  of  their  products. 

There's  an  alnost  linitless  anount  of  literature 
available,  docunenting  every  aspect  of  conputer  nanual 
developnent.  Information  on  suggested  content  and  fomat 
can  be  extremely  detailed,  and  is  often  specific  to  the  type 
of  manual  being  produced.  Although  the  degree  of  quality 
reviews  required  varies  from  company  to  company,  there  is  a 
growing  trend  in  the  literature  supporting  usability  testing 
before  publication.  This  usability  testing  is  similar  to 
the  verification  procedure  required  for  Air  Force  TOs.  The 
area  of  updating  and  controlling  software  user  manuals  once 
they're  published  has  received  the  least  attention,  but  good 
source  information  is  available. 

The  literature  available  on  different  procedures  used 
by  the  commercial  industry  demonstrates  that  many  companies 
are  imposing  much  more  detailed  and  stringent  requirements 
on  the  development  of  their  software  user  manuals,  than  the 
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Air  Force  currently  imposes  on  the  development  of  software 
user  manuals  for  its  own  use. 

In  general,  commercially  produced  software  user  manuals 
have  been  improving  over  the  last  decade.  The  recognition 
of  technical  writers  as  a  respected  profession,  the 
increased  competition  within  the  computer  industry,  and  an 
increased  awareness  and  publication  of  prior  deficiencies 
with  user  manuals,  have  all  contributed  to  improvements  in 
development  practices.  An  added  boon  is  that  companies  are 
sharing  their  experiences  and  publishing  articles  that 
detail  development  procedures.  All  the  information  needed 
to  set  up  a  quality  software  development  effort  is 
available,  waiting  to  be  consolidated. 

Smnmary 

This  chapter  used  information  presented  in  the 
Literature  Review  of  Chapter  III  to  answer  the  following 
research  questions: 

1.  What  are  all  the  regulations  guiding  the 
acquisition  and  development  of  Air  Force  software  user 
manuals? 

2.  How  do  the  requirements  for  Air  Force  software  user 
manuals  compare  to  the  requirements  for  Air  Force  Technical 
Orders? 

3.  How  do  the  requirements  for  Air  Force  software  user 
manuals  compare  to  best  commercial  practices  for  developing 
similar  manuals? 
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The  findings  indicate  that  the  requirements  for  Air 
Force  software  user  manuals  are  dispersed,  unclear,  and  non¬ 
specific.  The  requirements  for  Air  Force  Technical  Orders 
on  the  other  hand,  are  consolidated,  well  organized,  and 
extremely  detailed.  Although  the  requirements  for 
developing  and  producing  software  user  manuals  in  the 
commercial  industry  can  vary  from  company  to  company,  there 
are  many  detailed,  well  documented  procedures  available  for 
producing  quality  manuals. 

The  next  chapter  will  also  use  the  results  of  the 
Literature  Review  of  Chapter  III  to  emswer  the  last  research 
question: 

4.  What  improvements,  if  any,  can  be  made  to  the 
current  Air  Force  regulations  guiding  software  user  manual 
acquisition? 

In  addition,  recommendations  for  further  research  on 
this  issue,  as  well  as  related  topic  areas  affecting 
software  manual  development,  will  be  presented. 
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Chapter  V.  Conclusions  and  Recommendations 


Introduction 

This  chapter  will  sununarize  the  results  of  the 
literature  review  to  answer  the  last  research  question: 

4.  What  improvenents ,  if  any,  can  be  made  to  the 
current  Air  Force  regulations  guiding  software  user  manual 
acquisition? 

Recommendations  will  also  be  made  regarding  areas  for 
further  research,  and  related  topic  areas. 

Conclusions 

The  literature  review  presented  in  Chapter  III  clearly 
shows  the  need  for  improvements  in  the  acquisition 
rec[uirements  for  Air  Force  software  manuals.  The 
requirements  for  software  user  manuals  desperately  need 
consolidation!  The  option  which  first  comes  to  mind  is  the 
idea  of  including  software  user  manuals  in  the  TO 
requirements  system.  The  TO  system  is  centralized,  well 
organized,  and  highly  specific;  an  excellent  example  of 
quality  documentation  standards.  Including  software  manuals 
in  this  well  structured  system  could  have  distinct 
advantages.  The  requirements  for  TOs  are  primarily  found  in 
the  TO-00-5  series  of  regulations  which  are  well  established 
and  familiar  to  most  acquisition  organizations;  including 
software  manuals  in  this  series  of  regulations  would  provide 
a  localized,  and  well  recognized,  source  of  standards.  The 


44 


requirements  for  the  controlling  and  updating  of  software 
and  its  related  documentation  is  already  included  in  TO-00- 
5>16,  US  Air  Force  Computer  Program  Identification  Numbering 
System.  In  addition  to  localizing  the  requirements,  there 
are  many  techniques  used  for  developing  TOs  that  could  be 
applied  to  software  user  manuals.  For  example,  the 
requirements  for  validating  (contractor  checks  TO  against 
system)  and  verifying  (user  testing  for  accuracy  and 
readability)  TOs  would  provide  an  excellent  quality 
assurance  program  if  applied  to  software  user  manual 
development.  The  requirement  for  standardizing  the  reading 
grade  level  of  TOs  could  also  be  applied  to  software  user 
manuals.  Probably  the  greatest  disadvantage,  however,  is 
the  problem  of  adding  to  an  already  burdened  system  of 
regulations.  Because  the  requirements  for  all  the  various 
types  of  TOs  are  so  exacting,  the  regulations  for  their 
development  and  control  are  quite  extensive.  Adding 
software  user  manual  requirements  to  these  established 
standards  runs  two  risks:  the  requirements  for  software 
manuals  will  become  so  dispersed  throughout  the  regulations 
that  the  benefits  of  centralization  will  be  lost,  and  that 
unique  requirements  for  software  mi.  luals  may  be  overlooked 
when  fit  into  the  TO  regulation  structure. 

A  better  suggestion  is  to  create  a  separate  set  of 
regulations  which  are  specific  for  software  manuals.  The 
need  for  centralization  would  be  fulfilled,  and  the 
resulting  standards  would  potentially  be  much  better 
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organized  and  more  easily  referenced  than  a  huge  series  of 
combined  regulations.  This  would  simplify  the  job  of  Air 
Force  personnel  responsible  for  contracting  and 
administering  the  acquisition  of  these  software  user 
manuals,  helping  to  eliminate  mistakes  or  oversights  and 
contributing  to  better  quality  products.  This  doesn't  mean 
the  valuable  techniques  and  lessons  presented  in  the  TO 
system  of  regulations  should  be  overlooked.  There  are  many 
areas  of  dual  applicability  that  could  be  included  in  the 
separate  set  of  standards.  The  very  fact  of  their 
separation  would  increase  flexibility  for  modifying  TO 
requirements  if  necessary,  to  meet  specific  needs  of 
software  user  manuals.  The  system  for  developing  Air  Force 
TOs  has  evolved  and  improved  over  many  years,  and  should 
serve  as  a  first  template  for  organizing  and  developing  a 
set  of  unique  standards  for  software  manual  acquisition. 

The  quality  of  the  documentation  will  only  be  as 
good  as  the  intrinsic  quality  implied  by  the  standard. 
An  ambiguous  or  weak  standard  will  result  in 
documentation  of  dubious  quality.  (22:WE-52) 

It's  also  important  to  consider  the  techniques 

currently  being  used  by  private  industry  for  developing 

software  manuals.  The  incentives  for  profit  and  customer 

satisfaction  have  led  to  great  improvements  in  the  quality 

of  software  documentation  being  produced  in  the  commercial 

sector.  The  Air  Force  would  do  well  to  capitalize  on  those 

improvements  by  imposing  requirements  on  their  contractors 

which  parallel  or  duplicate  some  of  the  better  techniques 
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currently  in  use  in  private  industry.  For  example,  the  Air 
Force  could  require  their  contractors  to  hire  trained 
technical  writers,  document  their  system  of  internal 
reviews,  or  present  a  schedule  of  development  for  successive 
drafts  and  their  corresponding  reviews.  A  publications  plan 
prepared  by  the  contractor  and  submitted  to  the  Air  Force 
for  review  could  incorporate  all  of  the  above  issues;  the 
standards  for  a  Technical  Manual  Publications  Plan  (TMPP) 
and  a  Technical  Manual  Status  and  Schedule  (TMSS)  already 
exist,  and  are  required  for  the  accpiisition  of  Air  Force 
TOs.  The  requirement  for  a  Maintenance  Plan  similar  to  the 
one  discussed  in  Chapter  III  would  also  be  highly  desirable; 
one  of  the  enduring  facts  of  life  is  that  software  is  always 
modified,  and  it's  important  that  the  documentation  to 
support  that  software  be  updated  accordingly. 

Recommendations  for  Further  Research 

In  attempting  to  narrow  the  focus  of  this  research  and 
maintain  a  concise,  logical  flow  of  information,  many  issues 
related  to  the  acquisition  and  development  of  software  user 
manuals  were  excluded.  Perhaps  one  of  the  better  known 
issues  is  the  use  of  electronic  publishing  for  producing 
documentation.  This  topic  has  received  a  great  deal  of 
attention  in  the  commercial  literature,  and  is  definitely  a 
technique  that  is  gaining  popularity.  The  benefits  that 
electronic  publishing  could  have  on  the  organization  and 
potential  usability  of  software  manuals  were  beyond  the 
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scope  of  this  research,  but  deserve  to  be  evaluated.  Such 
an  evaluation  could  provide  the  Air  Force  with  valuable 
information  for  developing  future  acquisition  strategies  for 
software  user  manuals. 

In  developing  a  new  set  of  standards  for  software  user 
manuals,  it  is  strongly  suggested  that  inputs  from  the 
acquisition  personnel  who  will  use  the  regulations,  as  well 
as  the  end  users  of  the  manuals,  be  gathered  to  determine 
perceived  shortfalls  for  the  system.  Air  Force  users  are 
analogous  to  customers  in  the  private  sector;  whether  trying 
to  turn  a  profit,  or  increase  the  performance  of  the 
organization  (as  is  the  case  in  the  Air  Force),  customer 
satisfaction  is  the  key.  A  study  surveying  acquisition 
personnel  and  users  within  the  various  commands  would  be 
invaluable  for  ensuring  a  comprehensive  set  of  standards. 

This  research  has  examined  computer  user  manuals  in 
general  terms,  including  operating  manuals  and  maintenance 
manuals  in  the  same  group,  because  the  Air  Force  doesn't 
provide  separate  requirements  for  each.  Another  aspect  of 
developing  standards  for  computer  user  manuals  is  to  clearly 
define  the  requirements  for  both  types  of  user 
documentation;  many  times,  only  minor  aspects  of  software 
maintenance  are  allowed  because  it's  assumed  that  Air  Force 
maintenance  personnel  don't  have  the  expertise  to  perform 
any  detailed  maintenance  actions.  When  it  comes  to 
modifying  or  rewriting  lines  of  code,  the  assumption  is 
probably  correct;  however,  there  may  be  many  other  aspects 
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of  software  maintenance  that  could  be  performed  by  Air  Force 
personnel.  Considering  the  huge  costs  associated  with 
software  modifications,  it  would  be  a  good  idea  to  establish 
realistic  limits  on  the  amount  of  software  maintenance  that 
can  be  performed  by  Air  Force  personnel,  and  impose  those 
limits  Air  Force  wide. 

In  performing  the  research  to  develop  the  literature 
review  of  Chapter  III,  many  articles  were  reviewed  and 
determined  to  contain  information  related  to,  but  not 
directly  relevant  to,  the  aspects  of  the  problem  chosen  for 
discussion.  Appendix  A  has  been  included  to  provide  the 
reader  with  some  additional  sources  related  to  the  subject 
of  this  research,  but  not  cited  in  the  bibliography.  This 
list  is  not  meant  to  be  comprehensive;  the  area  of 
commercial  literature  is  so  extensive,  in  fact,  that 
Appendix  A  is  only  a  hint  at  the  amount  of  information 
available  regarding  computer  manual  issues. 

Summary 

This  chapter  has  used  the  information  presented  in  the 
literature  review  of  Chapter  III  to  answer  the  final 
research  question: 

4.  What  improvements,  if  any,  can  be  made  to  the 
current  Air  Force  regulations  guiding  software  user  manual 
acquisition? 

Based  on  the  results  of  the  literature  review,  it's 
clear  that  there  is  generous  room  for  improvement  in  the 
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system  of  regulations  guiding  Air  Force  software  user  manual 
acquisition.  Two  possible  solutions  are  immediately 
apparent:  combine  the  requirements  for  software  user  manuals 
with  those  for  technical  orders  or  develop  a  separate  set  of 
standards  for  the  software  user  manuals.  In  examining  some 
of  the  advantages  and  disadvantages  of  each  solution,  a 
recommendation  for  a  separate  set  of  regulations  is  the 
preferred  choice. 

C.QncluaiQn 

This  thesis  has  examined  in  detail,  the  regulations 
guiding  the  development  and  acquisition  of  Air  Force 
software  user  manuals.  It  has  also  compared  those 
regulations  to  the  requirements  for  developing  Air  Force 
Technical  Orders,  and  the  techniques  used  by  private 
industry  for  developing  similar  software  manuals.  Based  on 
the  results  of  this  research,  it  is  recommended  that  the  Air 
Force  revise  and  consolidate  the  regulations  for  developing 
software  manuals,  using  the  Technical  Order  system  as  a 
template. 
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Appendix  A;  Related  Information  Sources 


Military  Sources 

AFR  700  regulation  series,  Comrounications-Computer 
Systems. 

Bausman ,  Karen  B . ,  Computer  Program  Documentation- 
What  is  Qyerkill?.  Wright-Patterson  AFB  OH  (AD- 
P000198). 

Crawford,  Major  Nettle  L. ,  and  Capt  Edward  R.  Dawson, 
Ayionics  Intermediate  Shop  Software.  Air  Force 
Logistics  Management  Center,  Gunter  AFB  AL,  July  1989 
(Report  LM870733)  (AD-B135340) . 

Hall,  John  F.  II,  Documentation  for  Software 
Maintenance.  MS  thesis,  Nayal  Postgraduate  School, 
Monterey  CA,  December  1983  (AD-A138663) . 


Commercial  Sources 

Anderson,  Melissa  and  Ginny  Mahan,  "Usability  Testing: 
What  It  Is  and  What  It  Can  Do  for  Your  Documentation," 
Proceedings of  the  38th  International  Technical 
Commxinication  Conference.  New  York  NY,  April  1991. 

Bailey,  J.  and  S.  Pearson,  "Deyelopment  of  a  Tool  for 
Measuring  and  Analyzing  Computer  User  Satisfaction," 
Management  Science.  29(5):  530-545,  (1983). 

Bradford,  Dayid,  "Interaction  and  Interyention :  The 
Eyolution  of  Computer  Documentation  Standards," 
Proceedings  of  the  35th  International  Technical 
Communication  Conference.  Philadelphia  PA,  May  1988. 

Couse,  Mary  Margaret,  "Developing  an  Online  Information 
System  From  "Decision"  to  "Delivery","  Proceedings  of 
the  38th  International  Technical  Communication 
Conference.  New  York  NY,  April  1991. 

Daniel,  Margaret  A.,  "Getting  Users  Involved  in  the 
Documentation  Feedback  Process,"  Proceedings  of  the 
35th  International  Technical  Communication  Conference. 
Philadelphia  PA,  May  1988. 
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Dorazio,  Patricia  and  Freya  Y.  Winsberg,  "Usability 
Testing  of  Online  Information,"  Proceedings  of  the 
35th  International  Technical  Communication 
Conference.  Philadelphia  PA,  May  1988. 

Doyle,  Kelly  M.  and  Patricia  L.  Jones,  "Modularizing 
Software  Documentation,"  Proceedings  of  the  35th 
International  Technical  Communication  Conference. 
Philadelphia  PA,  May  1988. 

Horton,  William,  "Why  Most  Online  Documentation 
Systems  Pail  and  How  to  Make  Sure  Yours  Does  Not," 
Proceedings  of  the  38th  International  Technical 
Communication  Conference.  New  York  NY,  April  1991. 

Walker,  Janet  H. ,  "Issues  and  Strategies  for  Online 
Documentation,"  IEEE  Transactions  on  Professional 
Communication.  PC30M^;  235-238  (December  1987). 

Wheeler,  Kyle,  "Establishing  Standards  for  Computer 
Documentation,"  Proceedings  of  the  35th  International 
Technical  Communication  Conference.  Philadelphia  PA, 
May  1988. 
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